Technologies for audiovisual communication using interestingness algorithms

ABSTRACT

Technologies for audiovisual communication include an audiovisual server and a number of audiovisual client devices, including a presenter device and a number of audience devices. Each audience device captures an audiovisual stream and transmits the audiovisual stream to the audiovisual server. Each audience device also captures sensor input data such as eye tracking data or facial expression data and transmits abstracted sensor input data to the audiovisual server. The abstracted sensor input data may be based on the captured audiovisual stream. The audiovisual server determines an interestingness rating associated with each audience device based on the sensor input data, and selects one or more audiovisual streams based on the interestingness ratings. The audiovisual server transmits the selected audiovisual streams to the presenter device. The audiovisual server may update the interestingness rating algorithm based on feedback from the presenter device or the audience devices. Other embodiments are described and claimed.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of U.S. application Ser.No. 14/317,607, entitled “TECHNOLOGIES FOR AUDIOVISUAL COMMUNICATIONUSING INTERESTINGNESS ALGORITHMS,” which was filed on Jun. 27, 2014.

BACKGROUND

Video teleconferencing is a popular application for modern computingdevices, allowing globally distributed individuals to work, learn, andsocialize together. Compared to audio or text communicationtechnologies, video communication may provide better opportunities forvisual cues and other important human interactions. Typical videoteleconferencing systems may identify which party to a teleconference isspeaking based on audio levels of the transmitted signal. Whendisplaying the parties, the speaker may be enlarged or otherwiseemphasized, and other parties to the teleconference may be displayeduniformly or otherwise deemphasized. For teleconferences with a largenumber of participants such as presentations, each party other than thespeaker may be difficult to view in detail or otherwise interact with.

Interestingness is a concept used in the field of data mining andknowledge discovery. Processing large amounts of data may involveapplying data mining algorithms to identify patterns within the data andthen focusing further analysis only on “interesting” patterns amongthose identified. Interestingness is thus a measure of which patternsidentified within the data are interesting. In particular, aninterestingness measure may indicate the relevance, accuracy, orrepresentativeness of a particular pattern in relation to a data set.Various techniques and algorithms for measuring interestingness aresurveyed in Sigal Sahar, Interestingness Measures: On Determining WhatIs Interesting, in Data Mining and Knowledge Discovery Handbook (OdedMaimon & Lior Rokach eds., 2d ed. 2010).

BRIEF DESCRIPTION OF THE DRAWINGS

The concepts described herein are illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. Where considered appropriate, referencelabels have been repeated among the figures to indicate corresponding oranalogous elements.

FIG. 1 is a simplified block diagram of at least one embodiment of asystem for audiovisual communication;

FIG. 2 is a simplified block diagram of at least one embodiment ofvarious environments that may be established by the system of FIG. 1;

FIG. 3 is a simplified flow diagram of at least one embodiment of amethod for capturing and displaying audiovisual streams that may beexecuted by an audiovisual client device of FIGS. 1 and 2; and

FIG. 4 is a simplified flow diagram of at least one embodiment of amethod for analyzing, selecting, and transmitting audiovisual streamsthat may be executed by an audiovisual server of FIGS. 1 and 2.

DETAILED DESCRIPTION OF THE DRAWINGS

While the concepts of the present disclosure are susceptible to variousmodifications and alternative forms, specific embodiments thereof havebeen shown by way of example in the drawings and will be describedherein in detail. It should be understood, however, that there is nointent to limit the concepts of the present disclosure to the particularforms disclosed, but on the contrary, the intention is to cover allmodifications, equivalents, and alternatives consistent with the presentdisclosure and the appended claims.

References in the specification to “one embodiment,” “an embodiment,”“an illustrative embodiment,” etc., indicate that the embodimentdescribed may include a particular feature, structure, orcharacteristic, but every embodiment may or may not necessarily includethat particular feature, structure, or characteristic. Moreover, suchphrases are not necessarily referring to the same embodiment. Further,when a particular feature, structure, or characteristic is described inconnection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to effect such feature, structure,or characteristic in connection with other embodiments whether or notexplicitly described. Additionally, it should be appreciated that itemsincluded in a list in the form of “at least one A, B, and C” can mean(A); (B); (C): (A and B); (B and C); or (A, B, and C). Similarly, itemslisted in the form of “at least one of A, B, or C” can mean (A); (B);(C): (A and B); (B and C); or (A, B, and C).

The disclosed embodiments may be implemented, in some cases, inhardware, firmware, software, or any combination thereof. The disclosedembodiments may also be implemented as instructions carried by or storedon a transitory or non-transitory machine-readable (e.g.,computer-readable) storage medium, which may be read and executed by oneor more processors. A machine-readable storage medium may be embodied asany storage device, mechanism, or other physical structure for storingor transmitting information in a form readable by a machine (e.g., avolatile or non-volatile memory, a media disc, or other media device).

In the drawings, some structural or method features may be shown inspecific arrangements and/or orderings. However, it should beappreciated that such specific arrangements and/or orderings may not berequired. Rather, in some embodiments, such features may be arranged ina different manner and/or order than shown in the illustrative figures.Additionally, the inclusion of a structural or method feature in aparticular figure is not meant to imply that such feature is required inall embodiments and, in some embodiments, may not be included or may becombined with other features.

Referring now to FIG. 1, in the illustrative embodiment, a system 100for video conferencing includes a number of audiovisual client devices102 and an audiovisual server 108 capable of communication over anetwork 110. The audiovisual client devices 102 further may beclassified by role into an audiovisual presenter device 104 and a numberof audiovisual audience devices 106. In use, as described below, each ofthe audiovisual client devices 102 captures an audiovisual stream andtransmits the audiovisual stream to the audiovisual server 108. Each ofthe audiovisual client devices 102 also captures sensor input data andtransmits the sensor input data to the audiovisual server 108. Theaudiovisual server 108 generates an interestingness rating based on theincoming sensor input data for each of the audiovisual audience devices106, and selects one or more audiovisual streams corresponding toaudiovisual audience devices 106 having high interestingness ratings.The audiovisual server 108 transmits the selected audiovisual streams tothe audiovisual presenter device 104. The audiovisual server 108 mayrefine or otherwise improve the interestingness rating algorithm basedon sensor input data received from the audiovisual presenter device 104and/or the audiovisual audience devices 106. In some embodiments, theaudiovisual server 108 may transmit an audiovisual stream received fromthe audiovisual presenter device 104 to each of the audiovisual audiencedevices 106.

Thus, the audiovisual presenter device 104 may display audiovisualstreams from a relatively small number of interesting audiovisualaudience devices 106, allowing the presenter to view those audiencemembers in greater detail. Thus, the presenter may become aware ofvisual and/or other nonverbal cues displayed by some audience membersand adjust the presentation accordingly. For example, the presenter maybe able to gauge audience sentiment more readily, identify engaged ordisengaged audience members, or otherwise improve the quality of thepresentation.

Each audiovisual client device 102 may be embodied as any type of devicefor performing the functions described herein. For example, eachaudiovisual client device 102 may be embodied as, without limitation, asmartphone, a tablet computer, a laptop computer, a notebook computer, amobile computing device, a wearable computing device, a cellulartelephone, a handset, a messaging device, a vehicle telematics device, aserver computer, a desktop computer, a workstation, a distributedcomputing system, a multiprocessor system, a consumer electronic device,and/or any other computing device configured to perform the functionsdescribed herein. The illustrative audiovisual client devices 102include a processor 120, an input/output subsystem 122, a memory 124,and a data storage device 126. Of course, each audiovisual client device102 may include other or additional components, such as those commonlyfound in a computer (e.g., various input/output devices), in otherembodiments. Additionally, in some embodiments, one or more of theillustrative components may be incorporated in, or otherwise form aportion of, another component. For example, the memory 124, or portionsthereof, may be incorporated in the processor 120 in some embodiments.

The processor 120 may be embodied as any type of processor capable ofperforming the functions described herein. For example, the processor120 may be embodied as a single or multi-core processor(s), digitalsignal processor, microcontroller, or other processor orprocessing/controlling circuit. Similarly, the memory 124 may beembodied as any type of volatile or non-volatile memory or data storagecapable of performing the functions described herein. In operation, thememory 124 may store various data and software used during operation ofthe audiovisual client device 102 such as operating systems,applications, programs, libraries, and drivers. The memory 124 iscommunicatively coupled to the processor 120 via the I/O subsystem 122,which may be embodied as circuitry and/or components to facilitateinput/output operations with the processor 120, the memory 124, andother components of the audiovisual client device 102. For example, theI/O subsystem 122 may be embodied as, or otherwise include, memorycontroller hubs, input/output control hubs, firmware devices,communication links (i.e., point-to-point links, bus links, wires,cables, light guides, printed circuit board traces, etc.) and/or othercomponents and subsystems to facilitate the input/output operations. Insome embodiments, the I/O subsystem 122 may form a portion of asystem-on-a-chip (SoC) and be incorporated, along with the processor120, the memory 124, and other components of the audiovisual clientdevice 102, on a single integrated circuit chip.

The data storage device 126 may be embodied as any type of device ordevices configured for short-term or long-term storage of data such as,for example, memory devices and circuits, memory cards, hard diskdrives, solid-state drives, or other data storage devices. The datastorage device 126 may store audiovisual data or sensor input datacaptured by the audiovisual client device 102, or audiovisual data to bepresented by the audiovisual client device 102.

Each audiovisual client device 102 further includes a communicationcircuit 128, which may be embodied as any communication circuit, device,or collection thereof, capable of enabling communications between theaudiovisual client devices 102, the audiovisual server 108, and/or otherremote devices. The communication circuit 128 may be configured to useany one or more communication technology (e.g., wireless or wiredcommunications) and associated protocols (e.g., Ethernet, Bluetooth®,Wi-Fi®, WiMAX, etc.) to effect such communication. The communicationcircuit 128 may be embodied as a network adapter, including a wirelessnetwork adapter.

In the illustrative embodiment, each audiovisual client device 102further includes a display 130, audio device 132, audio input 134,camera 136, and a number of sensors 138. The display 130 of theaudiovisual client device 102 may be embodied as any type of displaycapable of displaying digital information such as a liquid crystaldisplay (LCD), a light emitting diode (LED), a plasma display, a cathoderay tube (CRT), or other type of display device. In some embodiments,the display 130 may be coupled to a touch screen to allow userinteraction with the audiovisual client device 102. The audio device 132may be embodied as any device capable of generating audio signals foroutput, such as a paper cone speaker, an audio transducer, an audiooutput jack, a digital-to-analog converter (DAC), or other type of audiodevice.

The audio input 134 may be embodied as any sensor capable of capturingaudio signals such as a microphone, a line input jack and associatedcircuitry, an analog-to-digital converter (ADC), or other type of audiosensor. The camera 136 may be embodied as a digital camera or otherdigital imaging device integrated with the audiovisual client device 102or otherwise communicatively coupled thereto. The camera 136 includes anelectronic image sensor, such as an active-pixel sensor (APS), e.g., acomplementary metal-oxide-semiconductor (CMOS) sensor, or acharge-coupled device (CCD). The camera 136 may be used to capture stillor video images of the environment of the audiovisual client device 102,including, in some embodiments, capturing the user of the audiovisualclient device 102. For example, the camera 136 of the audiovisualpresenter device 104 may capture the presenter, and the camera 136 ofeach audiovisual audience device 106 may capture an audience member.Thus, in some embodiments the audio input 134 and the camera 136 maycooperate to capture an audiovisual stream suitable forvideoconferencing.

The sensors 138 are capable of generating sensor input data indicativeof the environment of the audiovisual client device 102. Accordingly,the sensors 138 may be capable of detecting sensor input data relatingto the state of a user of the audiovisual client device 102, includingthe user's attention level, presence, or other indication of engagementwith the audiovisual client device 102. In some embodiments, the sensors138 may include, be coupled to, or be embodied as other components ofthe audiovisual client device 102, such as the audio input 134 and/orthe camera 136. In some embodiments, the sensors 138 may include an eyetracking sensor 140 and a facial expression sensor 142. Additionally oralternatively, the sensors 138 may include wearable devices, biometricdevices, and other devices capable of measuring the state of the user.

The eye tracking sensor 140 may be embodied as any one or more active orpassive sensors capable of determining a direction in which the user'sgaze is directed. For example, in some embodiments, the eye trackingsensor 140 may use active infrared emitters and infrared detectors totrack the viewer's eye movements over time. The eye tracking sensor 140may capture the infrared light reflected off of various internal andexternal features of the viewer's eye and thereby calculate thedirection of the viewer's gaze. In other embodiments, the eye trackingsensor 140 may be embodied as a video camera capable of recording theuser's eye motion. In some embodiments, the eye tracking sensor 140 maygather eye tracking data for both of the user's eyes to improve trackingaccuracy. In those embodiments, the eye tracking sensor 140 may use morethan one active or passive sensor components to track both of the user'seyes. Additionally or alternatively, in some embodiments the eyetracking sensor 140 may perform three-dimensional eye tracking, whichtracks both the gaze direction of the user's eye along with the distanceat which the user's eye is focused. For example, the eye tracking sensor140 may determine a viewing angle for both of the user's eyes, allowingthe distance to the object to be calculated.

In some embodiments, the sensors 138 may include the facial expressionsensor 142. The facial expression sensor 142 is capable of detecting andclassifying facial expressions of the user of the audiovisual clientdevice 102. For example, the facial expression sensor 142 may analyzedata from the camera 136 and/or other sensors 138 to determine thefacial expressions. The facial expression sensor 142 may be embodied ashardware, firmware, software, or any other component of the audiovisualclient device 102 capable of analyzing facial expressions. Additionallyor alternatively the sensors 138 may include other sensors to supplementor substitute for the eye tracking sensor 140 and/or the facialexpression sensor 142. For example, the sensors 138 may include atemperature sensor that may be used, for example, to help determine theuser's mood, in some embodiments in connection with a detected facialexpression.

The audiovisual server 108 may be embodied as any type of device forprocessing audiovisual streams as described herein. For example, theaudiovisual server 108 may be embodied as, without limitation, a servercomputer, a workstation, a desktop computer, a smart phone, a tabletcomputer, a laptop computer, a notebook computer, a mobile computingdevice, a cellular telephone, a handset, a messaging device, a vehicletelematics device, a distributed computing system, a multiprocessorsystem, a consumer electronic device, and/or any other computing deviceconfigured to perform the functions described herein. Further, theaudiovisual server 108 may be embodied as a single server computingdevice or a collection of servers and associated devices. For example,in some embodiments, the audiovisual server 108 is embodied as a cloudservice to perform the functions described herein. In such embodiments,the audiovisual server 108 may be embodied as a “virtual server” formedfrom multiple computing devices distributed across the network 110 andoperating in a public or private cloud. Accordingly, although theaudiovisual server 108 is illustrated in FIG. 1 and described below asembodied as single server computing device, it should be appreciatedthat the audiovisual server 108 may be embodied as multiple devicescooperating together to facilitate the functionality described below.

As shown in FIG. 1, the illustrative audiovisual server 108 may includecomponents and features similar to the audiovisual client devices 102,such as a processor 180, I/O subsystem 182, memory 184, data storage186, communication circuitry 188, and various peripheral devices. Thoseindividual components of the audiovisual server 108 may be similar tothe corresponding components of each audiovisual client device 102, thedescription of which is applicable to the corresponding components ofthe audiovisual server 108 and is not repeated for clarity of thepresent description.

Referring now to FIG. 2, in some embodiments each audiovisual clientdevice 102 establishes an environment 200 during operation. Theillustrative environment 200 includes an audiovisual capture module 202,a sensor control module 206, and a display module 210. In someembodiments, the environment 200 may also include an interestingnessmodule 224, described below. The various modules of the environment 200may be embodied as hardware, firmware, software, or a combinationthereof.

The audiovisual capture module 202 is configured to capture one or moreaudiovisual streams. For example, the audiovisual capture module 202 mayuse the camera 136 and the audio input 134 to record a video stream ofthe user of the audiovisual client device 102. The audiovisual streamsare provided to the sensor control module 206 and to the audiovisualserver 108, as described below. In some embodiments, the audiovisualcapture module 202 may be configured to zoom or otherwise focus theaudiovisual stream. As described below, the audiovisual client device102 may receive instructions to focus the audiovisual stream on aparticular subject from the audiovisual server 108. In some embodiments,those functions may be performed by a sub-module, for example by a zoomcontrol module 204.

The sensor control module 206 is configured to capture sensor input datafrom the sensors 138, and to analyze and regulate the captured sensorinput data. The sensor control module 206 may also capture, correlate,or otherwise receive captured audiovisual streams from the audiovisualcapture module 202. The sensor control module 206 may filter, process,or otherwise transform the sensor input data into a format that may beused for additional analysis. For example, the sensor control module 206may generate sensor data at a higher level of abstraction suitable forfurther analysis. The regulated sensor input data may be provided to theaudiovisual server 108 and/or, in some embodiments, the interestingnessmodule 224 of the audiovisual client device 102. In some embodiments,the sensor control module 206 may be configured to focus, guide, orotherwise direct the sensors 138 and/or the audiovisual capture module202 to focus on a particular subject. As described below, theaudiovisual client device 102 may receive instructions to focus thesensor input data on a particular subject from the audiovisual server108. In some embodiments, those functions may be performed by asub-module, for example by a sensor guidance module 208.

The display module 210 is configured to receive one or more selectedaudiovisual streams from the audiovisual server 108 and to display theselected audiovisual streams. The audiovisual streams may be selectedbased on an interestingness rating determined by the audiovisual server108 and/or the interestingness module 224 of the audiovisual clientdevice 102, as described below. The particular audiovisual streamsselected may depend on the role of each audiovisual client device 102;for example, the audiovisual audience devices 106 may each receive anaudiovisual stream from the audiovisual presenter device 104, and theaudiovisual presenter device 104 may receive one or more audiovisualstreams from audiovisual audience devices 106 having highinterestingness ratings.

Still referring to FIG. 2, in some embodiments, the audiovisual server108 establishes an environment 220 during operation. The illustrativeenvironment 220 includes a mixer module 222 and an interestingnessmodule 224. The various modules of the environment 220 may be embodiedas hardware, firmware, software, or a combination thereof.

The interestingness module 224 is configured to determine aninterestingness rating associated with an audiovisual client device 102based on the associated sensor input data and/or audiovisual streams. Insome embodiments, interestingness ratings may be determined for theaudiovisual audience devices 106 and not the audiovisual presenterdevice 104. The interestingness rating may provide an indication of therelevance, accuracy, or representativeness of the associated audiovisualstream. The interestingness module 224 is further configured to rank theaudiovisual client devices 102 based on interestingness. Additionally,the interestingness module 224 is configured to update theinterestingness algorithm based on feedback received from theaudiovisual client devices 102. The interestingness module 224 maymaintain interestingness history data 232 that may be used to update theinterestingness algorithm. In some embodiments, those functions may beperformed by one or more sub-modules, for example by a rating module226, a ranking module 228, or a feedback module 230.

As described above, in some embodiments the environment 200 may alsoinclude the interestingness module 224 and/or the interestingnesshistory data 232. In those embodiments, the interestingness module 224may perform the same functions in both the environment 200 and theenvironment 220. Additionally or alternatively, the functions of theinterestingness module 224 may be partially or completely shared orotherwise distributed between the environment 200 and the environment220.

The mixer module 222 is configured to receive audiovisual streams fromthe audiovisual client devices 102. The mixer module 222 is alsoconfigured to select one or more audiovisual streams for each of theaudiovisual client devices 102 based on the interestingness ratings andto transmit the selected audiovisual streams to the appropriateaudiovisual client devices 102. For example, the mixer module 222 maytransmit an audiovisual stream received from the audiovisual presenterdevice 104 to each of the audiovisual audience devices 106, and maytransmit one or more audiovisual streams received from audiovisualaudience devices 106 having high interestingness ratings to theaudiovisual presenter device 104.

Referring now to FIG. 3, in use, each audiovisual client device 102 mayexecute a method 300 for audiovisual communication. The method 300 maybe performed by either the audiovisual presenter device 104 or any ofthe audiovisual audience devices 106. The various processes of themethod 300 may be performed in different orders in some embodiments. Themethod 300 begins in block 302, in which the audiovisual client device102 captures an audiovisual input stream. The audiovisual stream may beembodied in any format suitable for network transmission and/or display.The audiovisual stream may be captured using the camera 136 and theaudio input 134. In some embodiments, in block 304 the audiovisualclient device 102 may focus on a subject based on instructions receivedfrom the audiovisual server 108. The audiovisual client device 102 mayperform any technique to focus, direct, or otherwise improve the qualityof the captured audiovisual stream in relation to the specified subject.For example, the audiovisual client device 102 may focus, zoom, orotherwise direct the camera 136 toward an identified speaker in order toenlarge or improve the quality of the captured audiovisual stream.

In block 306, the audiovisual client device 102 transmits the capturedaudiovisual input stream to the audiovisual server 108. As describedbelow, the audiovisual input stream may be further transmitted to one ormore other audiovisual client devices 102. In some embodiments, theaudiovisual client device 102 may transmit additional or alternativeaudiovisual streams. For example, an audiovisual presenter device 104may also transmit a slideshow, screen capture, presentation notes, orother supplemental content to the audiovisual server 108.

In block 308, the audiovisual client device 102 captures sensor inputdata using the sensors 138. The sensor input data may be indicative ofthe state of a user of the audiovisual client device 102 (e.g., thepresenter or an audience member) or, in some embodiments, of the generalenvironment of the audiovisual client device 102 (e.g., backgroundnoise). For example, the sensor input data may be indicative of theuser's facial expression such as whether the user is smiling, eyetracking information associated with the user, an emotional reaction ofthe user, or any other parameter of the state of the user. Additionallyor alternatively, in some embodiments the audiovisual client device 102may capture sensor data indicative of the state of several users thatare all using the same audiovisual client device 102. In someembodiments, in block 310, the audiovisual client device 102 may captureeye tracking sensor data from the eye tracking sensors 140. The eyetracking sensor data may be indicative of the direction of the user'sgaze. For example, the eye tracking sensor data may indicate that theuser's gaze is directed at a particular point on the display 130 andthus at a particular audiovisual stream displayed at that point.Additionally or alternatively, the eye tracking sensor data may indicatewhether the user's gaze is directed at the display 130 or directedelsewhere. In some embodiments, in block 312, the audiovisual clientdevice 102 may capture facial expression data using the facialexpression sensors 142. Of course in some embodiments, the audiovisualclient device 102 may capture the facial expression data using thecamera 136 without the use of a distinct facial expression sensor 142.In some embodiments, in block 314, the audiovisual client device 102 maycapture intonation and/or pitch information relating to the speech ofthe user. The audiovisual client device 102 may capture the intonationand/or pitch information using the audio input 134 or any other audioinput and processing components. In some embodiments, in block 316, theaudiovisual client device 102 may capture temperature data. Thetemperature level may be indicative of the temperature of the user ofthe audiovisual client device 102 and may thus also be indicative of themood, alertness, or other attributes of the user. The temperature datamay be captured using a temperature sensor, thermal imager, biometricdevice, or any other device capable of determining temperature.

In some embodiments, in block 318, the audiovisual client device 102 mayfocus the sensors 138 on a subject based on instructions received fromthe audiovisual server 108. Similar to focusing the audiovisual inputstream, the audiovisual client device 102 may perform any technique tofocus, direct, or otherwise improve the quality of the sensor input datarelation to the specified subject. For example, the audiovisual clientdevice 102 may focus, zoom, or otherwise direct the eye tracking sensors140 toward an identified speaker in order to enlarge or improve thequality of the sensor input data.

After capturing the sensor input data, in block 320 the audiovisualclient device 102 analyzes and abstracts the sensor input data. Theaudiovisual client device 102 may filter, process, or otherwisetransform the sensor input data into abstracted sensor input data thatmay be used by the audiovisual server 108, as described below. Theabstracted sensor input data may be embodied as a representation of thesensor input data that is smaller in size, at a higher granularity, orat a higher level abstraction. For example, in some embodiments, theaudiovisual client device 102 may process raw sensor input data andgenerate sensor data at a higher level of abstraction suitable forfurther analysis. For example, the audiovisual client device 102 maysmooth or otherwise filter eye tracking sensor data to removehigh-frequency eye movements. In some embodiments, the audiovisualclient device 102 may process raw video data to generate data indicativeof facial expressions, for example, data describing the position of theuser's facial features, or symbolic representations of facialexpressions (e.g., smile, frown, confusion, interest, etc.).

In block 322, the audiovisual client device 102 transmits the regulatedsensor input data to the audiovisual server 108 for analysis, asdescribed further below. In some embodiments, as described below, thesensor input data may be used by the audiovisual server 108 to determinean interestingness level associated with the audiovisual client device102. For example, the audiovisual server 108 may analyze sensor inputdata received from the audiovisual audience devices 106 to rank theaudiovisual audience devices 106 in order of interestingness.Additionally or alternatively, in some embodiments the sensor input datamay be used by the audiovisual server 108 as feedback to improve itsinterestingness algorithm. As another example, the audiovisual server108 may analyze sensor input data received from the audiovisualpresenter device 104 to determine whether the user's attention level isconsistent with the interestingness ranking assigned to each selectedaudiovisual stream.

In block 324, the audiovisual client device 102 receives one or moreselected audiovisual streams from the audiovisual server 108. Each ofthe audiovisual streams may originate from a different audiovisualclient device 102. For example, the audiovisual presenter device 104 mayreceive one or more audiovisual streams corresponding to audiovisualaudience devices 106 having relatively high interestingness ratings.Accordingly, each audiovisual audience device 106 may receive anaudiovisual stream originating from the audiovisual presenter device104. In some embodiments, the selected audiovisual stream may be one ofseveral audiovisual streams originating from the audiovisual presenterdevice 104 (e.g., a video feed, slideshow stream, screen capture stream,presentation notes, or other alternate audiovisual stream).

In block 326, the audiovisual client device 102 displays the selectedaudiovisual streams on the display 130. The audiovisual client device102 may display the audiovisual streams in any suitable format,including tiling the audiovisual streams, cascading the audiovisualstreams, or sequentially displaying the audiovisual streams. In someembodiments, in block 328 the audiovisual client device 102 may displayan indication of the interestingness rating associated with theaudiovisual stream and/or its associated audiovisual client device 102.For example, the audiovisual client device 102 may superimpose a visualindicator of interestingness such as a bar code, color-coding, ornumerical display on the audiovisual stream. As another example, theaudiovisual client device 102 may emphasize particular audiovisualstreams based on interestingness, for example by displaying the mostinteresting audiovisual stream with the largest size or in the center ofthe display 130. After displaying the selected audiovisual streams, themethod 300 loops back to block 302 to continue capturing the audiovisualstream.

Referring now to FIG. 4, in use, the audiovisual server 108 may executea method 400 for audiovisual communication. The method 400 begins withblock 402, in which the audiovisual server 108 receives an audiovisualstream captured by each of the audiovisual client devices 102. Theaudiovisual stream may include, for example, a video feed captured bythe camera 136 and the audio input 134 of the audiovisual client device102. Additionally or alternatively, the audiovisual server 108 mayreceive additional or alternative audiovisual streams from theaudiovisual client devices 102. For example, the audiovisual server 108may also receive a slideshow, screen capture, presentation notes, orother supplemental content from the audiovisual presenter device 104.

In block 404, the audiovisual server 108 receives sensor input data fromthe sensor control module 206 of each audiovisual client device 102. Asdescribed above, the sensor input data may include eye tracking data,facial expression data, intonation or pitch data, temperature data, orany other sensor data indicative of the environment of the audiovisualclient device 102 or the user of the audiovisual client device 102.

In block 406, the audiovisual server 108 rates an interestingness valueassociated with each of the audiovisual audience devices 106. Theinterestingness value may represent how interesting the particularaudiovisual stream associated with the audiovisual audience device 106is predicted to be for the user of the audiovisual presenter device 104.The interestingness value thus may represent a determination of theaccuracy, relevance, or representativeness of the particular audiovisualaudience device 106. For example, a high interestingness value mayindicate whether the associated audience member is attentive, excited,or otherwise engaged with the presenter, and a low interestingness valuemay indicate that the audience member is distracted, absent, orotherwise not engaged. As another example, a high interestingness valuemay indicate that the audience member is highly representative of otheraudience members (i.e., the audience member displays the same level ofengagement as many other audience members). As still another example,interestingness values may indicate whether the audiovisual stream isrelevant to the presentation (e.g., audiovisual streams including loudbackground noise may not be relevant). Additionally or alternatively,the audiovisual server 108 may determine an interestingness value formultiple audience members using a single audiovisual audience device106. The audiovisual server 108 may use any interestingness algorithm todetermine the interestingness value, including subjectiveinterestingness algorithms, objective interestingness algorithms,impartial interestingness algorithms, or any other interestingnessalgorithm.

In some embodiments, in block 408 the audiovisual server 108 maytransmit instructions to one or more audiovisual client devices 102 tofocus the sensor input data to improve the interestingness analysis. Forexample, the audiovisual server 108 may determine that the confidencelevel associated with a particular interestingness value is low, or thatthe interestingness value may otherwise be improved with better sensorinput data. If the sensor input data may be improved, the audiovisualserver 108 may generate instructions to focus the sensor input.

Although illustrated as occurring on the audiovisual server 108, in someembodiments the interestingness rating may be generated entirely orpartially on one or more of the audiovisual client devices 102. Forexample, each audiovisual client device 102 may entirely or partiallydetermine its own interestingness rating and submit that interestingnessrating (or partial interestingness rating) to the audiovisual server108.

In block 410, the audiovisual server 108 ranks the audiovisual audiencedevices 106 by their associated interestingness ratings. The audiovisualserver 108 may use any appropriate ranking. For example, the audiovisualserver 108 may rank the audiovisual audience devices 106 in ascending ordescending order.

In block 412, the audiovisual server 108 selects one or more audiovisualstreams for transmission to each audiovisual client device 102 based onthe associated interestingness rating. In some embodiments, in block414, the audiovisual server 108 may select one or more audiovisualstreams associated with the highest-ranking audiovisual audience devices106 for transmission to the audiovisual presenter device 104. Selectingthe audiovisual audience devices 106 with the highest interestingnessrating may allow the presenter to view the most accurate, relevant, orrepresentative audiovisual streams from the audiovisual audience devices106. Additionally or alternatively, in some embodiments the audiovisualserver 108 may select one or more audiovisual streams associated withthe lowest-ranking audiovisual audience devices 106 (not shown). Forexample, selecting the audiovisual audience devices 106 with the lowestinterestingness rating may allow an instructor to monitor audiencemembers that are confused, inattentive, or otherwise not engaged withthe presentation.

In some embodiments, in block 416, the audiovisual server 108 may selecta presenter stream based on the interestingness rating associated witheach of the audiovisual audience devices 106. For example, as discussedabove, the interestingness rating associated with the audiovisualaudience device 106 may indicate that the audience member is confused,inattentive, or disengaged. In those circumstances, the audiovisualserver 108 may select an alternative or supplemental presenter stream(e.g., a slideshow, screen capture, presentation notes, or other stream)for transmission to the audiovisual audience device 106.

In block 418, the audiovisual server 108 transmits the selectedaudiovisual streams to each of the audiovisual client devices 102. Asdescribed above, the audiovisual server 108 may transmit one or moreaudiovisual streams to the audiovisual presenter device 104 that areassociated with the audiovisual audience devices 106 having the highestinterestingness ratings. Further, the audiovisual server 108 maytransmit one or more audiovisual streams from the audiovisual presenterdevice 104 to each of the audiovisual audience devices 106.

In block 420, the audiovisual server 108 updates the interestingnessrating algorithm based on feedback received from the audiovisual clientdevices 102. The audiovisual server 108 may store historicalinterestingness data, including previous interestingness rankings andassociated sensor input data, for use in updating the interestingnessrating algorithm.

In block 422, the audiovisual server 108 updates the interestingnessalgorithm based on historical interestingness data associated with thepresenter. The audiovisual server 108 may analyze the sensor input dataand/or audiovisual streams originating from the audiovisual presenterdevice 104 to determine whether the presenter agrees with theinterestingness ratings determined by the interestingness algorithm. Insome embodiments, in block 424, the audiovisual server 108 may determinean attention level of the presenter associated with each selected A/Vstream. For example, the audiovisual server 108 may analyze eye trackingsensor data from the audiovisual presenter device 104 to determine thepresenter directed his or her gaze at a selected audiovisual stream.Next, in block 426, in those embodiments the audiovisual server 108 maycompare the determined presenter attention level with theinterestingness ranking associated with each selected audiovisualstream. For example, the audiovisual server 108 may determine that thepresenter primarily focused his or her gaze on a particular audiovisualstream, but that the particular audiovisual stream was ranked lower thanother audiovisual streams. The audiovisual server 108 may update theinterestingness rating algorithm based on that comparison.

In block 428, the audiovisual server 108 updates the interestingnessalgorithm based on historical interestingness data associated with theaudience members. For example, the interestingness algorithms may beadjusted based on particular facial expressions, gestures, intonation orpitch information, or other sensor input data received from theaudiovisual audience devices 106. Updating the interestingness algorithmbased on historical audience data may allow features associated withinterestingness (e.g., facial expressions, gestures, eye focus patterns,etc.) to be identified automatically or emergently. Additionally oralternatively, updating the interestingness algorithm based onhistorical audience data may account for differences in audience cultureor composition (e.g., adapting to cultural differences between audiencesfrom different countries).

In some embodiments, in block 430, the audiovisual server 108 may updatethe interestingness algorithm based on historical presenter and/oraudience data that originated from other communication sessions. Theaudiovisual server 108 may update the interestingness algorithm based ondata associated with prior communication sessions that involve the samepresenter and/or audience members. Additionally or alternatively, theaudiovisual server 108 may update the interestingness algorithm based ondata associated with any prior communication sessions, includingsessions not involving the current presenter and/or audience members.After updating the interestingness rating algorithm, the method 400loops back to block 402 to continue receiving audiovisual streams fromthe audiovisual client devices 102.

Examples

Illustrative examples of the technologies disclosed herein are providedbelow. An embodiment of the technologies may include any one or more,and any combination of, the examples described below.

Example 1 includes an audiovisual server for audiovisual communication,the audiovisual server comprising a mixer module to receive a pluralityof audiovisual streams from a plurality of audiovisual audience devices,wherein each audiovisual stream is captured by a correspondingaudiovisual audience device; and an interestingness module to receivesensor input data from each of the plurality of audiovisual audiencedevices, the sensor input data indicative of a state of a user of thecorresponding audiovisual audience device; determine, using aninterestingness algorithm, an interestingness rating associated witheach audiovisual audience device based on the sensor input data; andselect an audiovisual stream of the plurality of audiovisual streamsbased on the interestingness rating associated with each audiovisualclient device; wherein the mixer module is further to transmit theselected audiovisual stream to an audiovisual presenter device.

Example 2 includes the subject matter of Example 1, and wherein theinterestingness module is further to rank the plurality of audiovisualaudience devices based on the associated interestingness rating; andwherein to select the audiovisual stream comprises to select anaudiovisual stream captured by an audiovisual audience device that hasthe highest interestingness rating.

Example 3 includes the subject matter of any of Examples 1 and 2, andwherein the interestingness rating comprises an accuracy value, arelevancy value, or representativeness value associated with eachaudiovisual audience device.

Example 4 includes the subject matter of any of Examples 1-3, andwherein the interestingness rating is indicative of an engagement levelof a user of the audiovisual audience device.

Example 5 includes the subject matter of any of Examples 1-4, andwherein the sensor input data comprises eye tracking data, facialexpression data, voice pitch data, voice intonation data, or temperaturedata.

Example 6 includes the subject matter of any of Examples 1-5, andwherein the interestingness module is further to determine whetheraccuracy of the interestingness rating of a corresponding audiovisualaudience device could be improved by focusing on a particular parameterof the state of the user; and transmit instructions to the correspondingaudiovisual audience device to focus the sensor input data on theparticular parameter of the user in response to a determination that theaccuracy of the interestingness rating may be improved.

Example 7 includes the subject matter of any of Examples 1-6, andwherein the interestingness module is further to update theinterestingness algorithm based on the interestingness ratings.

Example 8 includes the subject matter of any of Examples 1-7, andwherein to update the interestingness algorithm comprises to receivesensor input data from the audiovisual presenter device, the sensorinput data indicative of a physical environment of the audiovisualpresenter device; and update the interestingness algorithm based on thesensor input data received from the audiovisual presenter device.

Example 9 includes the subject matter of any of Examples 1-8, andwherein the interestingness module is further to select a secondaudiovisual stream of the plurality of audiovisual streams based on theinterestingness rating associated with each audiovisual client device;the mixer module is further to transmit the second audiovisual stream tothe audiovisual presenter client device; and to update theinterestingness algorithm comprises to determine a presenter attentionlevel associated with each of the selected audiovisual stream and thesecond audiovisual stream, the presenter attention level based on thesensor input data received from the audiovisual presenter device;compare the presenter attention level associated with each of theselected audiovisual stream and the second audiovisual stream with theinterestingness rating associated with each audiovisual client device;and update the interestingness algorithm based on a comparison betweenthe presenter attention level and the interesting rating.

Example 10 includes the subject matter of any of Examples 1-9, andwherein to update the interestingness algorithm further comprises toupdate the interestingness algorithm based on the sensor input datareceived from each of the plurality of audiovisual audience devices.

Example 11 includes the subject matter of any of Examples 1-10, andwherein to update the interestingness algorithm comprises to update theinterestingness algorithm based on an interestingness rating associatedwith another audiovisual communication session.

Example 12 includes the subject matter of any of Examples 1-11, andwherein the other audiovisual communication session comprises a prioraudiovisual communication session of the audiovisual presenter device.

Example 13 includes the subject matter of any of Examples 1-12, andwherein the mixer module is further to receive a presenter audiovisualstream from the audiovisual presenter device; and transmit the presenteraudiovisual stream to the plurality of audiovisual audience devices.

Example 14 includes the subject matter of any of Examples 1-13, andwherein the presenter audiovisual stream is captured by the audiovisualpresenter device.

Example 15 includes the subject matter of any of Examples 1-14, andwherein the mixer module is further to receive a plurality of presenteraudiovisual streams from the audiovisual presenter device, the pluralityof presenter audiovisual streams to include the presenter audiovisualstream; and the interestingness module is further to determine apresenter interestingness rating associated with each presenteraudiovisual stream device based on the sensor input data, using theinterestingness algorithm; and select the presenter audiovisual streamof the plurality of presenter audiovisual streams based on the presenterinterestingness rating associated with each presenter audiovisualstream.

Example 16 includes the subject matter of any of Examples 1-15, andwherein the mixer module is further to transmit the interestingnessrating associated with the selected audiovisual stream to theaudiovisual presenter device.

Example 17 includes an audiovisual client device for audiovisualcommunication, the audiovisual client device comprising an audiovisualcapture module to (i) capture an audiovisual input stream and (ii)transmit the audiovisual input stream to an audiovisual server; and asensor control module to (i) capture sensor input data indicative of astate of a user of the audiovisual client device, (ii) analyze thesensor input data to generate abstracted sensor input datarepresentative of the sensor input data, and (iii) transmit theabstracted sensor input data to the audiovisual server.

Example 18 includes the subject matter of Example 17, and wherein theaudiovisual client device comprises an audiovisual audience device; andthe sensor control module is further to (i) receive instructions tofocus the sensor input data from the audiovisual server and (ii) focusthe sensor input data in response to receipt of the instructions.

Example 19 includes the subject matter of any of Examples 17 and 18, andwherein the sensor input data is indicative of an engagement level of auser of the audiovisual client device.

Example 20 includes the subject matter of any of Examples 17-19, andwherein the sensor input data comprises eye tracking sensor data, facialexpression sensor data, intonation pitch sensor data or temperaturedata.

Example 21 includes the subject matter of any of Examples 17-20, andwherein the audiovisual client device comprises an audiovisual presenterdevice, the audiovisual presenter device further comprising a displaymodule to receive an audiovisual stream from the audiovisual server, theaudiovisual stream to originate from an audiovisual audience device andbe selected based on an interestingness value associated with theaudiovisual audience device; and display the audiovisual stream.

Example 22 includes the subject matter of any of Examples 17-21, andwherein to display the audiovisual stream further comprises to displayan indication of the interestingness value associated with theaudiovisual audience device of the audiovisual stream.

Example 23 includes the subject matter of any of Examples 17-22, andfurther including an interestingness module to determine theinterestingness rating associated with the audiovisual audience deviceusing an interestingness algorithm.

Example 24 includes the subject matter of any of Examples 17-23, andwherein the interestingness module is further to update theinterestingness algorithm based on the abstracted sensor input data.

Example 25 includes the subject matter of any of Examples 17-24, andwherein the display module is further to receive a second audiovisualstream from the audiovisual server, the second audiovisual stream tooriginate from a second audiovisual audience device and be selectedbased on an interestingness value associated with the second audiovisualaudience device; and to update the interestingness algorithm comprisesto determine a presenter attention level associated with each of theaudiovisual stream and the second audiovisual stream, the presenterattention level based on the abstracted sensor input data; compare thepresenter attention level associated with each of the audiovisual streamand the second audiovisual stream with the interestingness ratingassociated with each audiovisual audience device; and update theinterestingness algorithm based on a comparison between the presenterattention level and the interesting rating.

Example 26 includes a method for audiovisual communication, the methodcomprising receiving, by an audiovisual server, a plurality ofaudiovisual streams from a plurality of audiovisual audience devices,wherein each audiovisual stream is captured by a correspondingaudiovisual audience device; receiving, by the audiovisual server,sensor input data from each of the plurality of audiovisual audiencedevices, the sensor input data indicative of a state of a user of thecorresponding audiovisual audience device; determining, by theaudiovisual server using an interestingness algorithm, aninterestingness rating associated with each audiovisual audience devicebased on the sensor input data; selecting, by the audiovisual server, anaudiovisual stream of the plurality of audiovisual streams based on theinterestingness rating associated with each audiovisual client device;and transmitting, by the audiovisual server, the selected audiovisualstream to an audiovisual presenter device.

Example 27 includes the subject matter of Example 26, and furtherincluding ranking, by the audiovisual server, the plurality ofaudiovisual audience devices based on the associated interestingnessrating; wherein selecting the audiovisual stream comprises selecting anaudiovisual stream captured by an audiovisual audience device that hasthe highest interestingness rating.

Example 28 includes the subject matter of any of Examples 26 and 27, andwherein determining the interestingness rating comprises determining anaccuracy value, a relevancy value, or representativeness valueassociated with each audiovisual audience device.

Example 29 includes the subject matter of any of Examples 26-28, andwherein determining the interestingness rating comprises determining aninterestingness rating indicative of an engagement level of a user ofthe audiovisual audience device.

Example 30 includes the subject matter of any of Examples 26-29, andwherein receiving the sensor input data comprises receiving eye trackingdata, facial expression data, voice pitch data, voice intonation data,or temperature data from the audiovisual client device.

Example 31 includes the subject matter of any of Examples 26-30, andfurther including determining, by the audiovisual server, whetheraccuracy of the interestingness rating of a corresponding audiovisualaudience device could be improved by focusing on a particular parameterof the state of the user; and transmitting, by the audiovisual server,instructions to an audiovisual audience device to focus the sensor inputdata on the particular parameter of the user in response to determiningthe accuracy of the interestingness rating may be improved.

Example 32 includes the subject matter of any of Examples 26-31, andfurther including updating, by the audiovisual server, theinterestingness algorithm based on the interestingness ratings.

Example 33 includes the subject matter of any of Examples 26-32, andwherein updating the interestingness algorithm comprises receiving, bythe audiovisual server, sensor input data from the audiovisual presenterdevice, the sensor input data indicative of a physical environment ofthe audiovisual presenter device; and updating the interestingnessalgorithm based on the sensor input data received from the audiovisualpresenter device.

Example 34 includes the subject matter of any of Examples 26-33, andfurther including selecting, by the audiovisual server, a secondaudiovisual stream of the plurality of audiovisual streams based on theinterestingness rating associated with each audiovisual client device;and transmitting, by the audiovisual server, the second audiovisualstream to the audiovisual presenter client device; wherein updating theinterestingness algorithm comprises: determining a presenter attentionlevel associated with each of the selected audiovisual stream and thesecond audiovisual stream, the presenter attention level based on thesensor input data received from the audiovisual presenter device;comparing the presenter attention level associated with each of theselected audiovisual stream and the second audiovisual stream with theinterestingness rating associated with each audiovisual client device;and updating the interestingness algorithm based on comparing thepresenter attention level and the interesting rating.

Example 35 includes the subject matter of any of Examples 26-34, andwherein updating the interestingness algorithm further comprisesupdating the interestingness algorithm based on the sensor input datareceived from each of the plurality of audiovisual audience devices.

Example 36 includes the subject matter of any of Examples 26-35, andwherein updating the interestingness algorithm comprises updating theinterestingness algorithm based on an interestingness rating associatedwith another audiovisual communication session.

Example 37 includes the subject matter of any of Examples 26-36, andwherein the other audiovisual communication session comprises a prioraudiovisual communication session of the audiovisual presenter device.

Example 38 includes the subject matter of any of Examples 26-37, andfurther including receiving, by the audiovisual server, a presenteraudiovisual stream from the audiovisual presenter device; andtransmitting, by the audiovisual server, the presenter audiovisualstream to the plurality of audiovisual audience devices.

Example 39 includes the subject matter of any of Examples 26-38, andwherein receiving the presenter audiovisual stream comprises receiving apresenter audiovisual stream captured by the audiovisual presenterdevice.

Example 40 includes the subject matter of any of Examples 26-39, andfurther including receiving, by the audiovisual server, a plurality ofpresenter audiovisual streams from the audiovisual presenter device, theplurality of presenter audiovisual streams to include the presenteraudiovisual stream; determining, by the audiovisual server, a presenterinterestingness rating associated with each presenter audiovisual streamdevice based on the sensor input data, using the interestingnessalgorithm; and selecting, by the audiovisual server, the presenteraudiovisual stream of the plurality of presenter audiovisual streamsbased on the presenter interestingness rating associated with eachpresenter audiovisual stream.

Example 41 includes the subject matter of any of Examples 26-40, andfurther including transmitting, by the audiovisual server, theinterestingness rating associated with the selected audiovisual streamto the audiovisual presenter device.

Example 42 includes a method for audiovisual communication, the methodcomprising capturing, by an audiovisual client device, an audiovisualinput stream; transmitting, by the audiovisual client device, theaudiovisual input stream to an audiovisual server; capturing, by anaudiovisual audience device, sensor input data indicative of a state ofa user of the audiovisual client device; analyzing, by the audiovisualclient device, the sensor input data to generate abstracted sensor inputdata representative of the sensor input data; and transmitting, by theaudiovisual client device, the abstracted sensor input data to theaudiovisual server.

Example 43 includes the subject matter of Example 42, and wherein theaudiovisual client device comprises an audiovisual audience device, themethod further comprising receiving, by the audiovisual audience device,instructions to focus the sensor input data from the audiovisual server;and focusing, by the audiovisual audience device, the sensor input datain response to receiving the instructions.

Example 44 includes the subject matter of any of Examples 42 and 43, andwherein capturing the sensor data comprises capturing sensor dataindicative of an engagement level of a user of the audiovisual audiencedevice.

Example 45 includes the subject matter of any of Examples 42-44, andwherein capturing the sensor input data comprises capturing eye trackingsensor data, facial expression sensor data, intonation pitch sensor dataor temperature data.

Example 46 includes the subject matter of any of Examples 42-45, andwherein the audiovisual client device comprises an audiovisual presenterdevice, the method further comprising receiving, by the audiovisualpresenter device, an audiovisual stream from the audiovisual server, theaudiovisual stream originating from an audiovisual audience device andselected based on an interestingness value associated with theaudiovisual audience device; and displaying, by the audiovisualpresenter device, the audiovisual stream.

Example 47 includes the subject matter of any of Examples 42-46, andwherein displaying the audiovisual stream further comprises displayingan indication of the interestingness value associated with theaudiovisual audience device of the audiovisual stream.

Example 48 includes the subject matter of any of Examples 42-47, andfurther including determining, by the audiovisual presenter device, theinterestingness rating associated with the audiovisual audience deviceusing an interestingness algorithm.

Example 49 includes the subject matter of any of Examples 42-48, andfurther including updating the interestingness algorithm based on theabstracted sensor input data.

Example 50 includes the subject matter of any of Examples 42-49, andfurther including receiving, by the audiovisual presenter device, asecond audiovisual stream from the audiovisual server, the secondaudiovisual stream originating from a second audiovisual audience deviceand selected based on an interestingness value associated with thesecond audiovisual audience device; and wherein updating theinterestingness algorithm comprises: determining a presenter attentionlevel associated with each of the audiovisual stream and the secondaudiovisual stream, the presenter attention level based on theabstracted sensor input data; comparing the presenter attention levelassociated with each of the audiovisual stream and the secondaudiovisual stream with the interestingness rating associated with eachaudiovisual audience device; and updating the interestingness algorithmbased on comparing the presenter attention level and the interestingrating.

Example 51 includes a computing device comprising a processor; and amemory having stored therein a plurality of instructions that whenexecuted by the processor cause the computing device to perform themethod of any of Examples 26-50.

Example 52 includes one or more machine readable storage mediacomprising a plurality of instructions stored thereon that in responseto being executed result in a computing device performing the method ofany of Examples 26-50.

Example 53 includes a computing device comprising means for performingthe method of any of Examples 26-50.

Example 54 includes an audiovisual server for audiovisual communication,the audiovisual server comprising means for receiving a plurality ofaudiovisual streams from a plurality of audiovisual audience devices,wherein each audiovisual stream is captured by a correspondingaudiovisual audience device; means for receiving sensor input data fromeach of the plurality of audiovisual audience devices, the sensor inputdata indicative of a state of a user of the corresponding audiovisualaudience device; means for determining, using an interestingnessalgorithm, an interestingness rating associated with each audiovisualaudience device based on the sensor input data; means for selecting anaudiovisual stream of the plurality of audiovisual streams based on theinterestingness rating associated with each audiovisual client device;and means for transmitting the selected audiovisual stream to anaudiovisual presenter device.

Example 55 includes the subject matter of Example 54, and furtherincluding means for ranking the plurality of audiovisual audiencedevices based on the associated interestingness rating; wherein themeans for selecting the audiovisual stream comprises means for selectingan audiovisual stream captured by an audiovisual audience device thathas the highest interestingness rating.

Example 56 includes the subject matter of any of Examples 54 and 55, andwherein the means for determining the interestingness rating comprisesmeans for determining an accuracy value, a relevancy value, orrepresentativeness value associated with each audiovisual audiencedevice.

Example 57 includes the subject matter of any of Examples 54-56, andwherein the means for determining the interestingness rating comprisesmeans for determining an interestingness rating indicative of anengagement level of a user of the audiovisual audience device.

Example 58 includes the subject matter of any of Examples 54-57, andwherein the means for receiving the sensor input data comprises meansfor receiving eye tracking data, facial expression data, voice pitchdata, voice intonation data, or temperature data from the audiovisualclient device.

Example 59 includes the subject matter of any of Examples 54-58, andfurther including means for determining whether accuracy of theinterestingness rating of a corresponding audiovisual audience devicecould be improved by focusing on a particular parameter of the state ofthe user; and means for transmitting instructions to an audiovisualaudience device to focus the sensor input data on the particularparameter of the user in response to determining the accuracy of theinterestingness rating may be improved.

Example 60 includes the subject matter of any of Examples 54-59, andfurther including means for updating the interestingness algorithm basedon the interestingness ratings.

Example 61 includes the subject matter of any of Examples 54-60, andwherein the means for updating the interestingness algorithm comprisesmeans for receiving sensor input data from the audiovisual presenterdevice, the sensor input data indicative of a physical environment ofthe audiovisual presenter device; and means for updating theinterestingness algorithm based on the sensor input data received fromthe audiovisual presenter device.

Example 62 includes the subject matter of any of Examples 54-61, andfurther including means for selecting a second audiovisual stream of theplurality of audiovisual streams based on the interestingness ratingassociated with each audiovisual client device; and means fortransmitting the second audiovisual stream to the audiovisual presenterclient device; wherein the means for updating the interestingnessalgorithm comprises: means for determining a presenter attention levelassociated with each of the selected audiovisual stream and the secondaudiovisual stream, the presenter attention level based on the sensorinput data received from the audiovisual presenter device; means forcomparing the presenter attention level associated with each of theselected audiovisual stream and the second audiovisual stream with theinterestingness rating associated with each audiovisual client device;and means for updating the interestingness algorithm based on comparingthe presenter attention level and the interesting rating.

Example 63 includes the subject matter of any of Examples 54-62, andwherein the means for updating the interestingness algorithm furthercomprises means for updating the interestingness algorithm based on thesensor input data received from each of the plurality of audiovisualaudience devices.

Example 64 includes the subject matter of any of Examples 54-63, andwherein the means for updating the interestingness algorithm comprisesmeans for updating the interestingness algorithm based on aninterestingness rating associated with another audiovisual communicationsession.

Example 65 includes the subject matter of any of Examples 54-64, andwherein the other audiovisual communication session comprises a prioraudiovisual communication session of the audiovisual presenter device.

Example 66 includes the subject matter of any of Examples 54-65, andfurther including means for receiving a presenter audiovisual streamfrom the audiovisual presenter device; and means for transmitting thepresenter audiovisual stream to the plurality of audiovisual audiencedevices.

Example 67 includes the subject matter of any of Examples 54-66, andwherein the means for receiving the presenter audiovisual streamcomprises means for receiving a presenter audiovisual stream captured bythe audiovisual presenter device.

Example 68 includes the subject matter of any of Examples 54-67, andfurther including means for receiving a plurality of presenteraudiovisual streams from the audiovisual presenter device, the pluralityof presenter audiovisual streams to include the presenter audiovisualstream; means for determining a presenter interestingness ratingassociated with each presenter audiovisual stream device based on thesensor input data, using the interestingness algorithm; and means forselecting the presenter audiovisual stream of the plurality of presenteraudiovisual streams based on the presenter interestingness ratingassociated with each presenter audiovisual stream.

Example 69 includes the subject matter of any of Examples 54-68, andfurther including means for transmitting the interestingness ratingassociated with the selected audiovisual stream to the audiovisualpresenter device.

Example 70 includes an audiovisual client device for audiovisualcommunication, the audiovisual client device comprising means forcapturing an audiovisual input stream; means for transmitting theaudiovisual input stream to an audiovisual server; means for capturingsensor input data indicative of a state of a user of the audiovisualclient device; means for analyzing the sensor input data to generateabstracted sensor input data representative of the sensor input data;and means for transmitting the abstracted sensor input data to theaudiovisual server.

Example 71 includes the subject matter of Example 70, and wherein theaudiovisual client device comprises an audiovisual audience device, theaudiovisual audience device further comprising means for receivinginstructions to focus the sensor input data from the audiovisual server;and means for focusing the sensor input data in response to receivingthe instructions.

Example 72 includes the subject matter of any of Examples 70 and 71, andwherein the means for capturing the sensor data comprises means forcapturing sensor data indicative of an engagement level of a user of theaudiovisual audience device.

Example 73 includes the subject matter of any of Examples 70-72, andwherein the means for capturing the sensor input data comprises meansfor capturing eye tracking sensor data, facial expression sensor data,intonation pitch sensor data or temperature data.

Example 74 includes the subject matter of any of Examples 70-73, andwherein the audiovisual client device comprises an audiovisual presenterdevice, the audiovisual presenter device further comprising means forreceiving an audiovisual stream from the audiovisual server, theaudiovisual stream originating from an audiovisual audience device andselected based on an interestingness value associated with theaudiovisual audience device; and means for displaying the audiovisualstream.

Example 75 includes the subject matter of any of Examples 70-74, andwherein the means for displaying the audiovisual stream furthercomprises means for displaying an indication of the interestingnessvalue associated with the audiovisual audience device of the audiovisualstream.

Example 76 includes the subject matter of any of Examples 70-75, andfurther including means for determining the interestingness ratingassociated with the audiovisual audience device using an interestingnessalgorithm.

Example 77 includes the subject matter of any of Examples 70-76, andfurther including means for updating the interestingness algorithm basedon the abstracted sensor input data.

Example 78 includes the subject matter of any of Examples 70-77, andfurther including means for receiving a second audiovisual stream fromthe audiovisual server, the second audiovisual stream originating from asecond audiovisual audience device and selected based on aninterestingness value associated with the second audiovisual audiencedevice; and wherein the means for updating the interestingness algorithmcomprises: means for determining a presenter attention level associatedwith each of the audiovisual stream and the second audiovisual stream,the presenter attention level based on the abstracted sensor input data;means for comparing the presenter attention level associated with eachof the audiovisual stream and the second audiovisual stream with theinterestingness rating associated with each audiovisual audience device;and means for updating the interestingness algorithm based on comparingthe presenter attention level and the interesting rating.

1. An audiovisual server for audiovisual communication, the audiovisualserver comprising: a mixer module to receive a plurality of audiovisualstreams from a plurality of audiovisual audience devices, wherein eachaudiovisual stream is captured by a corresponding audiovisual audiencedevice; and an interestingness module to: receive sensor input data fromeach of the plurality of audiovisual audience devices, the sensor inputdata indicative of a state of a user of the corresponding audiovisualaudience device; determine, using an interestingness algorithm, aninterestingness rating associated with each audiovisual audience devicebased on the sensor input data; and select an audiovisual stream of theplurality of audiovisual streams based on the interestingness ratingassociated with each audiovisual client device; wherein the mixer moduleis further to transmit the selected audiovisual stream to an audiovisualpresenter device.
 2. The audiovisual server of claim 1, wherein theinterestingness module is further to rank the plurality of audiovisualaudience devices based on the associated interestingness rating; andwherein to select the audiovisual stream comprises to select anaudiovisual stream captured by an audiovisual audience device that hasthe highest interestingness rating.
 3. The audiovisual server of claim1, wherein the interestingness rating comprises an accuracy value, arelevancy value, or representativeness value associated with eachaudiovisual audience device.
 4. The audiovisual server of claim 1,wherein the interestingness rating is indicative of an engagement levelof a user of the audiovisual audience device.
 5. The audiovisual serverof claim 1, wherein the interestingness module is further to: determinewhether accuracy of the interestingness rating of a correspondingaudiovisual audience device could be improved by focusing on aparticular parameter of the state of the user; and transmit instructionsto the corresponding audiovisual audience device to focus the sensorinput data on the particular parameter of the user in response to adetermination that the accuracy of the interestingness rating may beimproved.
 6. The audiovisual server of claim 1, wherein theinterestingness module is further to update the interestingnessalgorithm based on the interestingness ratings.
 7. The audiovisualserver of claim 6, wherein to update the interestingness algorithmcomprises to: receive sensor input data from the audiovisual presenterdevice, the sensor input data indicative of a physical environment ofthe audiovisual presenter device; and update the interestingnessalgorithm based on the sensor input data received from the audiovisualpresenter device.
 8. The audiovisual server of claim 7, wherein: theinterestingness module is further to select a second audiovisual streamof the plurality of audiovisual streams based on the interestingnessrating associated with each audiovisual client device; the mixer moduleis further to transmit the second audiovisual stream to the audiovisualpresenter client device; and to update the interestingness algorithmcomprises to: determine a presenter attention level associated with eachof the selected audiovisual stream and the second audiovisual stream,the presenter attention level based on the sensor input data receivedfrom the audiovisual presenter device; compare the presenter attentionlevel associated with each of the selected audiovisual stream and thesecond audiovisual stream with the interestingness rating associatedwith each audiovisual client device; and update the interestingnessalgorithm based on a comparison between the presenter attention leveland the interesting rating.
 9. The audiovisual server of claim 6,wherein to update the interestingness algorithm comprises to update theinterestingness algorithm based on an interestingness rating associatedwith another audiovisual communication session.
 10. The audiovisualserver of claim 1, wherein: the mixer module is further to: receive aplurality of presenter audiovisual streams from the audiovisualpresenter device, the plurality of presenter audiovisual streams toinclude a presenter audiovisual stream; and transmit the presenteraudiovisual stream to the plurality of audiovisual audience devices; andthe interestingness module is further to: determine a presenterinterestingness rating associated with each presenter audiovisual streamdevice based on the sensor input data, using the interestingnessalgorithm; and select the presenter audiovisual stream of the pluralityof presenter audiovisual streams based on the presenter interestingnessrating associated with each presenter audiovisual stream.
 11. Anaudiovisual client device for audiovisual communication, the audiovisualclient device comprising: an audiovisual capture module to (i) capturean audiovisual input stream and (ii) transmit the audiovisual inputstream to an audiovisual server; and a sensor control module to (i)capture sensor input data indicative of a state of a user of theaudiovisual client device, (ii) analyze the sensor input data togenerate abstracted sensor input data representative of the sensor inputdata, and (iii) transmit the abstracted sensor input data to theaudiovisual server.
 12. The audiovisual client device of claim 11,wherein: the audiovisual client device comprises an audiovisual audiencedevice; and the sensor control module is further to (i) receiveinstructions to focus the sensor input data from the audiovisual serverand (ii) focus the sensor input data in response to receipt of theinstructions.
 13. The audiovisual client device of claim 11, wherein theaudiovisual client device comprises an audiovisual presenter device, theaudiovisual presenter device further comprising a display module to:receive an audiovisual stream from the audiovisual server, theaudiovisual stream to originate from an audiovisual audience device andbe selected based on an interestingness value associated with theaudiovisual audience device; and display the audiovisual stream.
 14. Theaudiovisual presenter device of claim 13, further comprising aninterestingness module to determine the interestingness ratingassociated with the audiovisual audience device using an interestingnessalgorithm.
 15. The audiovisual presenter device of claim 14, wherein:the display module is further to receive a second audiovisual streamfrom the audiovisual server, the second audiovisual stream to originatefrom a second audiovisual audience device and be selected based on aninterestingness value associated with the second audiovisual audiencedevice; and the interestingness module is further to update theinterestingness algorithm based on the abstracted sensor input data,wherein to update the interestingness algorithm comprises to: determinea presenter attention level associated with each of the audiovisualstream and the second audiovisual stream, the presenter attention levelbased on the abstracted sensor input data; compare the presenterattention level associated with each of the audiovisual stream and thesecond audiovisual stream with the interestingness rating associatedwith each audiovisual audience device; and update the interestingnessalgorithm based on a comparison between the presenter attention leveland the interesting rating.
 16. One or more computer-readable storagemedia comprising a plurality of instructions that in response to beingexecuted cause an audiovisual server to: receive a plurality ofaudiovisual streams from a plurality of audiovisual audience devices,wherein each audiovisual stream is captured by a correspondingaudiovisual audience device; receive sensor input data from each of theplurality of audiovisual audience devices, the sensor input dataindicative of a state of a user of the corresponding audiovisualaudience device; determine, using an interestingness algorithm, aninterestingness rating associated with each audiovisual audience devicebased on the sensor input data; select an audiovisual stream of theplurality of audiovisual streams based on the interestingness ratingassociated with each audiovisual client device; and transmit theselected audiovisual stream to an audiovisual presenter device.
 17. Theone or more computer-readable storage media of claim 16, furthercomprising a plurality of instructions that in response to beingexecuted cause the audiovisual server to rank the plurality ofaudiovisual audience devices based on the associated interestingnessrating; wherein to select the audiovisual stream comprises to select anaudiovisual stream captured by an audiovisual audience device that hasthe highest interestingness rating.
 18. The one or morecomputer-readable storage media of claim 16, further comprising aplurality of instructions that in response to being executed cause theaudiovisual server to: determine whether accuracy of the interestingnessrating of a corresponding audiovisual audience device could be improvedby focusing on a particular parameter of the state of the user; andtransmit instructions to an audiovisual audience device to focus thesensor input data on the particular parameter of the user in response todetermining the accuracy of the interestingness rating may be improved.19. The one or more computer-readable storage media of claim 16, furthercomprising a plurality of instructions that in response to beingexecuted cause the audiovisual server to update the interestingnessalgorithm based on the interestingness ratings.
 20. The one or morecomputer-readable storage media of claim 19, further comprising aplurality of instructions that in response to being executed cause theaudiovisual server to: select a second audiovisual stream of theplurality of audiovisual streams based on the interestingness ratingassociated with each audiovisual client device; and transmit the secondaudiovisual stream to the audiovisual presenter client device; whereinto update the interestingness algorithm comprises to: receive sensorinput data from the audiovisual presenter device, the sensor input dataindicative of a physical environment of the audiovisual presenterdevice; determine a presenter attention level associated with each ofthe selected audiovisual stream and the second audiovisual stream, thepresenter attention level based on the sensor input data received fromthe audiovisual presenter device; compare the presenter attention levelassociated with each of the selected audiovisual stream and the secondaudiovisual stream with the interestingness rating associated with eachaudiovisual client device; and update the interestingness algorithmbased on comparing the presenter attention level and the interestingrating.